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Abstract 

Research  has  indicated  that  group  decision  making  participants  have 
difficulty  in  sharing  their  uniquely  held  information  and  in  integrating 
unique  information  from  other  participants.  An  on-line  knowledge 
elicitation  technique  is  proposed  where  participants  assign  a  variety  of 
cognitive  assessments  to  each  retained  decision-relevant  information  item. 
These  assessments  might  include  source  credibility,  timeliness, 
information  uncertainty,  direction  of  impact  on  the  related  decision 
criterion  as  well  as  overall  importance  of  the  information  item.  The 
assessments  are  automatically  converted  into  a  small  color-coded  icon 
called  an  Information  Object  (lOB).  The  lOB  can  then  be  used  in  place  of 
the  text  version  of  the  item,  much  as  tactical  symbology  replaces  raw  radar 
video.  The  display,  sharing,  manipulation  and  algorithmic  sorting  of  these 
lOBs  permits  all  participants  to  access  each  item  and  simplifies  the 
process  of  integrating  the  arriving  information  and  making  quality 
decisions.  The  attached  individual  information  quality  assessments  also 
permit  a  highly  focused  discussion  exchange,  reducing  the  time  required 
for  conflict  resolution  and  consensus  building. 


1.0  Introduction; 

In  any  group  decision-making  (DM)  task,  there  are  always  two  categories  of  decision¬ 
relevant  information.  There  is  Shared  Information  (SI),  i.e.  information  that  is  known  to 
all  the  DM  participants.  In  a  command  and  control  environment,  SI  information  items 
might  include  rules  of  engagement,  order  of  battle,  Commander’s  Intent,  the  OPLAN, 
etc.  In  a  business  environment,  these  might  include  items  such  as  CNN  reports, 
newspapers,  widely  circulated  company  documents,  ticker  quotes,  etc.  In  any  group, 
there  is  also  Unique  Information  (UI),  i.e.  decision-relevant  information  that  is  uniquely 
held  by  one  (or  some,  but  not  all)  of  the  DM  participants  and  not  known  to  the  remainder 
of  the  participants.  This  UI  can  result  from  either  a  specific  information  search  path  used 
by  a  participant  during  the  DM  task,  or  from  the  aggregate  of  information  the  participant 
has  acquired  from  his  past  experiences  and  contacts.  Examples  of  UI  could  include 
corporate  knowledge  held  by  a  former  employee,  findings  from  a  person  with  access  to  a 
restricted  database,  assessment  of  a  nation’s  infrastructure  by  a  recent  resident,  a  person 
capable  of  reading  foreign  language  newspapers/reports,  etc.  UI  impacts  the  pool  of 
information  by  either  adding  a  new  relevant  item  to  the  pool,  or  by  reflecting  (positively 
or  negatively)  on  the  credibility,  importance,  timeliness,  etc.,  of  information  items  that 
are  already  in  the  pool. 
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It  is  widely  assumed  that  (1)  UI  is  freely  shared  within  the  group  and  (2)  as  more  of  this 
information  is  made  available  to  the  group,  the  better  will  be  the  overall  quality  of  the 
decision  making,  i.e.  more  shared  UI  means  better  decisions. 

Problem  Area:  Extensive  research  has  been  conducted  on  the  two  issues  of  how  much  UI 
gets  shared  and  how  well  the  shared  UI  is  integrated  into  the  on-going  decision  process. 
The  early  and  classic  work  in  the  field  is  by  Stasser  and  Titus  (1985).  Participants  in  a 
face-to-face  (F2F)  environment  were  asked  to  review  hypothetical  job  applicants  and 
select  the  best  candidate.  In  one  condition,  each  participant  had  all  the  relevant 
information,  e.g.  if  there  were  5  items  to  consider  on  an  application,  each  participant  had 
all  5  items  in  front  of  them.  At  the  conclusion,  it  was  judged  that  this  group  had  made  the 
correct  decision  about  83%  of  the  time.  In  another  F2F  situation,  all  the  information  was 
“on  the  table”  but  was  distributed  across  the  participants  such  that  each  participant  had 
some  SI  items  as  well  as  some  UI  items,  but  no  one  had  all  of  the  information  items.  If 
all  the  UI  was  properly  shared,  this  would  be  exactly  the  same  as  the  first  condition.  It 
was  sharing  of  UI  did  not  happen  since  the  percent  of  correct  decisions  for  this  group 
dropped  to  24%.  Analysis  indicated  that  the  preponderance  of  discussion  time  was 
focused  on  the  SI,  and  insufficient  time  devoted  to  the  UI.  Stasser  and  Titus  (1985) 
referred  to  this  focus  on  SI  as  “biased  sampling”  and  Wittenbaum  et  al.  (1999)  expanded 
the  concept  by  including  a  “mutual  enhancement”  explanation  that  attributes  the 
increased  level  of  SI  discussion  to  the  fact  that  it  permits  group  participants  to  mutually 
confirm  each  member’s  expertise  in  the  DM  task.  This  finding  of  poor  group  sharing  of 
UI  has  been  replicated  in  a  number  of  other  studies  (for  a  review  see  Fleming  &  Kaiwi, 
2002) 

With  regard  to  the  integration  of  shared  UI  into  the  decision  process,  the  majority  of 
studies  have  produced  results  that  indicate  that  group  members  discount  the  UI  and 
typically  base  their  decision  on  the  original  shared  information.  As  one  researcher  in  this 
area  accurately  entitled  his  article,  “You  can  lead  a  group  to  information  but  you  can’t 
make  it  think”  (Dennis,  1996),  meaning,  of  course,  that  getting  all  the  relevant 
information  does  not  improve  decision  making  if  that  information  is  ignored.  Another 
group  of  researchers  (Lavery  et  al  1999)  concluded,  “the  vital  role  of  group  discussion 
was  not  to  exchange  information  but  to  aggregate  member  preferences  into  a  consensual 
group  judgment”.  A  large  number  of  research  results  indicates  that:  UI  is  not  effectively 
utilized  once  it  is  shared;  is  repeated  less  often  that  original  shared  information;  is 
recalled  less  accurately  than  SI;  and  generally  discounted  in  one  form  or  another.  Gigone 
and  Hastie  (1993)  have  referred  to  the  phenomena  of  over- weighting  SI  information  and 
undervaluing  the  UI  information  as  the  “common  knowledge  effect”.  Gigone  &  Hastie 
(1997b)  referring  to  their  1993  work,  noted,  “Information  pooled  during  discussion  had 
almost  no  effect  on  the  group  judgments.  It  was  as  if  the  group  members  exchanged  and 
combined  their  opinions  but  paid  little  attention  to  anything  else”. 

Two  basic  conclusions  can  be  drawn:  (1)  People  are  not  very  effective  in  communicating 
unshared  information—  groups  tend  to  focus  their  discussion  on  information  that  is 
already  shared,  with  the  result  that  little  unshared  information  moves  into  the  shared 
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environment  and  (2)  when  unshared  information  does  move  into  the  shared  environment, 
participants  tend  to  ignore  or  discount  this  information  and  not  factor  it  into  their  decision 
process  in  an  effective  manner.  The  net  result  is  that  many  group  decisions  are  based 
upon  incomplete  information,  information  that  would  be  available  to  the  group  if  they 
were  optimally  exchanging  and  integrating  unshared  information. 

The  majority  of  the  previously  cited  research  was  conducted  in  a  F2F  environment. 
Consequently  the  deficiencies  in  sharing  and  integrating  UI  are  generally  attributed  to 
social  factors,  e.g.  prestige  of  members,  personality  styles,  group  dynamics,  etc.,  as  well 
as  cognitive  factors.  In  contrast,  much  of  modem  military  command  and  control 
decision-making  is  not  predicated  on  F2F  groups,  but  rather  on  time/place  asynchronous 
collaboration  where  the  participants  are  distributed  over  time  and  location.  In  this 
asynchronous  collaboration,  information  is  generally  exchanged  via  text,  such  as  email, 
message  traffic,  chat  boxes,  etc.  These  communication  modes  tend  to  be  more 
impersonal  and  thus  tend  to  minimize  social  factors  that  can  impact  the  exchange  of  UI. 
Consequently,  we  believe  the  primary  obstacle  to  the  effective  sharing  and  utilization  of 
UI  in  asynchronous  collaboration  is  the  cognitive  burden  placed  on  both  the  sender  of  UI 
and  the  recipient  of  new  information.  The  sender  needs  to  answer  such  questions  as,  is 
the  item  indeed  unique,  is  it  the  best  example  to  send,  who  should  be  the  recipient  and 
when  should  it  be  sent?  The  recipient  has  the  task  of  determining  how  (or  if)  this  newly 
arriving  information  should  be  integrated  into  his  on-going  decision  process.  The 
cognitive  burden  of  these  tasks  reduces  the  likelihood  that  UI  will  be  shared  or  used.  The 
following  section  describes  a  conceptual  model  of  information  management  and 
exchange  that  is  directed  at  mitigating  these  deficiencies. 

2.0  Conceptual  Model 

Application  Area:  In  specifying  any  DM  research  plan,  the  first  step  is  to  define  in  detail 
the  parameters  of  the  particular  DM  task  to  be  investigated.  There  is  no  “one  size  fits  all” 
decision  support  system.  Our  proposed  target  decision-making  group  is  a  command  and 
control  staff  that  has  been  asked  to  make  an  option-selection  recommendation.  The 
primary  task  assigned  to  the  group  is  to  recommend,  as  a  group,  one  option  out  of  a  set  of 
two  or  more  options.  The  recommendation  can  range  from  a  binary  yes-no,  action-no 
action  choice  (“Does  this  situation  call  for  the  establishment  of  a  Joint  Task  Force?”)  to  a 
selection  from  multiple  courses  of  action  (COAs),  e.g.  “Should  we  use  Japan,  Okinawa  or 
Singapore  as  the  refueling  point?”  In  our  project  emphasis,  the  target  group  is  not 
dealing  with  a  brainstorming  task  that  involves  the  generation/creation  of  COAs,  but 
rather  is  required  to  collect  and  analyze  the  information  needed  to  select  the  best  option 
from  an  already  existing  list.  We  also  do  not  address  open-ended  “report”  type  tasks,  e.g. 
“What  are  the  likely  consequences  if  this  regional  government  loses  the  election?” 

Our  basic  assumptions  are  that  the  target  DM  group: 

1 .  Is  not  under  exceptionally  critical  time  pressure,  but  may  have  hours  or  days  to 
produce  a  product. 

2.  Works  asynchronously,  both  in  terms  of  time  and  place. 
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3.  Most  often  must  make  a  choice  where  there  is  no  inherently  correct  answer. 

4.  Is  comprised  of  individual  participants  who  collect  information  in  a  relatively 
independent  manner,  and  then  collaborate  in  order  to  reach  an  overall  consensus 
on  the  option  recommendation. 

5.  Participants  get  their  information  from  documents,  message  traffic,  specialized 
databases,  intranets  and  the  internet,  personal  contacts  and  past  experiences. 

6.  Utilizes  collaborative  technologies  that  are  primarily  textual,  i.e.  email  and  chat 
boxes,  but  may  also  make  use  of  audio  record  and  playback,  virtual  rooms, 
telephones  and  limited  VTC. 

7.  Has  no  specific  limitation  on  group  size,  but  for  practical  purposes  it  is  assumed 
the  group  size  would  be  ten  or  fewer  participants. 

8.  Is  relatively  homogeneous  in  terms  of  age,  rank,  education,  computer 
sophistication  and  broad  areas  of  expertise. 

Sample  Scenario:  To  explain  the  functioning  of  the  model,  a  sample  DM  task  has  been 
created.  The  sample  task  involves  several  command  staff  personnel  who  have  been  asked 
to  make,  on  a  five-point  scale,  a  yes-no  recommendation.  The  scenario  is  that  a  large 
typhoon  has  caused  major  damage  to  a  coastal  nation.  There  are  expected  to  be  a  large 
number  of  refugees.  Country  X  has  been  proposed  as  a  site  for  the  refugee  camp.  The 
question  posed  is  “Do  you  recommend  Country  X  as  the  refugee  camp  site?”  The  staff 
has  been  told  to  base  their  recommendation  on  four  decision  parameters:  (1)  the 
Infrastructure  of  Country  X,  i.e.  capability  of  its  roads,  airports,  docks,  etc.  (2)  the  Labor 
Pool  available,  i.e.  can  Country  X  adequately  staff  the  site  with  the  right  type  and  number 
of  people  to  run  the  camp  (3)  the  number  and  adequacy  of  the  local  Security  forces  that 
Country  X  can  assign  to  the  camp  and  (4)  the  Impact  that  this  choice  would  have  on  our 
own  resources.  The  staff  was  told  to  consider  each  of  these  criteria  of  equal  importance. 

It  was  assumed  that  each  group  member  worked  independently  until  they  felt  they  had 
collected  enough  data,  at  which  point  they  would  score  the  four  criteria  and  then  make 
their  overall  recommendation.  At  that  point  the  group  would  then  exchange  their  scoring 
profiles  and  address  conflict  resolution  issues  and  initiate  consensus  building,  to  arrive  at 
the  group  overall  recommendation  to  be  forwarded  to  the  original  requester. 

Information  Objects  (lOBs)  :  When  each  of  the  staff  personnel  start  their  information 
search,  they  go  through  a  subjective  filtering  process  to  determine  if  any  particular  item 
should  be  retained.  The  sequential  steps  of  this  process  include: 

1 .  Is  the  item  relevant?  Specifically,  does  it  relate  to  one  of  the  decision  criterion? 

2.  It  is  a  usable  item?  This  addresses  the  quality  of  the  information,  such  as  the 
source  credibility,  the  timeliness  of  the  information  and  the  confidence  in  the 
accuracy  of  the  information. 

3.  What  does  the  content  imply?  This  addresses  both  the  direction  and  degree  of 
effect  of  the  information  content.  Does  it  have  a  positive  or  negative  effect 
(direction)  on  the  criterion  and  how  strong  is  this  effect  (degree)? 

4.  How  important  is  this  item?  This  factor  addresses  the  relative  importance  of  this 
item  compared  with  other  items  that  have  been  retained  for  this  criterion.  It  is 
usually  determined  by  a  subjective  composite  of  information  quality  and  content. 
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Normally,  these  subjective  parameters  are  implicitly  assigned  to  the  item,  i.e.  they  are 
retained  as  part  of  the  internal  cognitive  process.  An  “information  object  (lOB)”  is 
created  when  these  parameters  are  explicitly  attached  to  the  item,  i.e.  a  knowledge 
elicitation  (KE)  technology  quantitatively  assigns  a  participant’s  subjective  score  to  each 
parameter,  a  score  that  then  becomes  a  permanent  part  of  the  information  item.  For 
example,  a  subjective  perception  that  “. .  .this  item  has  high  source  credibility”  becomes, 
through  KE,  a  selection  of  HIGH  on  a  LOW-HIGH  source  credibility  continuum 
displayed  to  the  decision  maker.  We  define  an  lOB  as  any  information  item  that  contains 
the  quantitative  cognitive  information  assessment  about  each  of  the  factors  described 
above.  A  sample  lOB  might  contain: 


Criterion: 

Credibility: 

Timeliness: 

Confidence: 

Direction: 

Degree: 

Importance: 

Keywords: 


Labor  Pool 

High 

Average 

Low 

Negative 

Mild 

High 

Strike  likely 


Here,  the  researcher  has  found  information  about  a  possible  workers  strike,  assigned  it  to 
the  Labor  Pool  criterion  and  assessed  it  as  a  highly  important  item  that  has  a  negative 
effect  on  this  criterion. 


Knowledge  Elicitation:  As  noted  earlier,  we  attribute  difficulties  in  processing  UI  to  the 
cognitive  burden  it  imposes.  Consequently,  if  we  are  going  to  ask  decision  makers  to  do 
a  new,  additional  task,  such  as  creating  lOBs,  it  must  exert  a  minimum  burden  so  that  its 
cognitive  cost  does  not  exceed  any  benefits  it  may  bring  to  the  decision  process.  This 
implies  that  the  KE  interface  must  be  very  simple,  easy  to  learn  and  user  friendly.  Figure 
1  shows  a  proposed  KE  template  (which  could  be  activated  by  a  right-hand  mouse  click). 
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Figure  1:  Sample  lOB  knowledge  elicitation  template. 

In  the  box  to  the  right  of  “SITE”  the  user  would  enter  a  hyperlinked  description  of  the 
item.  The  hyperlink  is  connected  to  the  original  document/web  page,  making  it  available 
at  any  time  for  a  full  review. 

The  green  areas  are  where  the  user  enters  his  scoring  selection  for  the  various  information 
parameters.  The  first  three  include  source  Credibility,  Timeliness  of  the  information,  and 
the  users’  perception  of  his  overall  Confidence  in  the  accuracy  of  the  information  (this 
could  also  be  referred  to  as  Uncertainty).  The  scale  is  1-5  (worst  to  best)  with  the  ?6? 
category  serving  as  an  “unknown”  or  “defer  till  later”  option. 

The  EFFECT  and  IMPORTance  parameters  are  the  major  inputs.  EFFECT  is  the 
perception  of  how  this  information  affects  the  decision  criterion,  i.e.  is  it  supportive 
(categories  4  or  5)  or  non- supportive  (categories  I  or  2)?  Word  descriptors  that  would  go 
with  the  1-5  scaling  are  Strong  Negative,  Negative,  Positive  and  Strong  Positive.  The 
mid-range  (3)  is  not  used  here  since  it  is  assumed  that  an  item  with  neutral/no  effect  on  a 
criterion  would  not  be  retained  by  the  user.  IMPORT  is  the  users  perception  of  the 
Importance  of  this  lOB  compared  with  the  other  lOBs  retained  for  this  decision  criterion. 

The  I  category  implies  “Discard”  and  would  be  primarily  used  when  assessing  another 
members  item.  The  3  is  Average  importance,  4  is  High  importance  and  5  is  Very  High 
importance. 

The  final  row  includes  the  decision  criterion  to  which  this  item  has  been  assigned  by  the 
user  (e.g.  INF,  for  Infrastructure),  the  participant’s  designation  (e.g.  member  #2,  M2)  and 
a  unique  designator  (e.g.  #1)  to  identify  this  item  from  all  others. 

Information  Object  (lOB)  Format:  The  major  purpose  of  the  template  scoring  is  to  use  the 
cognitive  assessments  to  automatically  create  an  lOB.  It  is  a  symbolic  representation  of  the 
analyst’s  assessment  of  the  item.  An  immediate  analogy  is  the  tactical  symbology  that  replaces 
raw  radar  video.  An  lOB  is  a  symbolic  representation  of  a  text  document.  As  currently 
formatted,  an  lOB  includes:  (I)  the  upper  bar  has  the  keyword  descriptor  that  has  been  selected 
by  the  analyst.  This  functions  as  a  quick  refresher  to  the  analyst  regarding  the  content  of  the 
item.  A  click  on  this  bar  area  will  hyperlink  to  the  complete  original  information  item  (2)  the 
center  bar  displays  the  importance  and  effect  of  the  item.  The  number  of  filled  rectangles  (0-3 
max)  indicates  the  importance  of  the  item,  while  the  color  of  the  rectangle  indicates  the  effect. 
There  are  two  negative  colors  (orange  and  red)  and  two  positive  colors  (light  blue  and  dark  blue). 
(3)  The  left  side  of  the  lower  bar  indicates  the  decision  criterion  to  which  this  item  has  been 
assigned  (INF),  the  ownership  of  the  object  (group  member  #2),  and  a  unique  designator 
assigned  by  the  computer  for  tracking  and  reference  purposes.  The  right  side  of  the  lower  bar 
shows  the  scoring  assigned  to  Credibility,  Timeliness  and  Confidence.  Figure  2  depicts  an  lOB 
and  the  scored  template  that  generated  it. 
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I]VFM2#1  □  □  HI 


Figure  2:  Completed  Template  and  the  lOB  it  generated. 


Figure  3  is  an  expansion  of  this  lOB  showing  the  retained  cognitive  parameter  assignments. 


This  hyperlinked  article, 
gives  Negative  support. ...  H 
to  the  INF  decision  criterion.^  INF  M2  #1 


Originator  is  Member  #2 

Refer  to  designator  #1 


Source  is  of  Good  credibility 

Average  Timeliness 


-  High  Importance 
(2  boxes  filled) 

Originator  has  low 
confidence  in  the  data 


Figure  3.  Decomposition  of  a  sample  lOB. 
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The  typical  computer  data  folder  contains  either  standard  Microsoft  icons  or  a  text  list  of  the 
documents.  If  such  a  folder  contains  all  the  INF  information  items,  one  could  not  form  an 
immediate  “overall  assessment  of  the  INF  criterion”  by  looking  at  this  type  of  information.  If 
however,  the  documents  are  replaced  by  their  lOB  equivalents,  immediate  perceptions  and 
assessments  are  readily  apparent.  Figure  5  shows  such  an  example,  and  clearly  a  very  quick 
perusal  indicates  a  somewhat  negative  (non- supportive)  assessment  of  the  INF  criterion. 


1  Uneitiployittent  predicted  to 

; Judge  Kamos  murder  on 

;fer)T  traffic  curtaile  d 

INFM2S15>  ■  mi  im 

INF  M2  #22  m  m  m 

INF  M2  #25  ■  □  n 

\  MipArt  construe  do  11  delays 

;  survey  shows  that 

1 - ^ - 1 

oil  production  is  up  fin  m 

1 - 1 - ^ - 1 

INF  M2  HI  □  @  □ 

1 _ ^ _ 1 

INF  M2  #24  ■  m  ■ 

1 _ 1 _ 1 _ 1 

INF  M2  #12  m  m  □ 

',]m\  of  literacy  has  risen 

1 - - 1 

:  Nunteiuus  arrests  in  the 

:  Tourist  traffic  into  area  is 

1 - ^ ^ - 1 

INF  M2  #32  ■  ■  m 

INF  M2  #21  ■  ■  m 

INFM2#2«  H  □  □ 

Figure  4.  Database/file  representation  using  lOBs 

The  list  of  lOBs  above  includes  all  the  Infrastructure  items  that  have  been  found  by  member  #2 
(M2).  There  are,  however,  other  group  members,  and  they  may  have  found  additional  lOBs.  M2 
can  request  that  all  non-duplicative  lOBs  from  other  members  be  transferred  to  his  database  to 
give  hin  additional  information  on  the  INF  criterion.  When  this  is  action  is  performed  by  all 
group  members,  there  are  no  longer  any  UI  items,  all  items  are  now  shared  and  accessed  by  all 
participants. 

3.0  Using  lOBs  for  Criterion  and  Overall  decision  recommendations. 

Each  group  member  has  a  finite  set  of  decisions  to  be  made,  one  for  each  criterion  and  then, 
grouping  all  the  criterion  decisions,  one  for  an  Overall  recommendation  regarding  Option  X.  A 
variety  of  lOB  sorting  algorithms  could  assist  in  this  decision  process,  e.g.  Sort  by  Importance, 
or  Effect,  or  Credibility,  or  by  some  composite  weighting  of  factors.  In  this  regard,  we  expect  to 
rely  heavily  on  the  Electronic  Card  Wall  (EWall)  project  team  supervised  by  Patrick  H.  Winston, 
William  Porter  and  Paul  Keel  at  the  Artificial  Intelligence  Lab  at  MIT. 

EWall:  EWall  is  a  computational  framework  that  enables  decision-makers  to  quickly 
view,  collect,  organize,  and  communicate  large  amounts  of  information  as  well  as  to 
facilitate  the  collaboration  of  decision-makers  with  different  levels  of  involvement  in 
large,  distributed  and  decentralized  teams  across  organizational  boundaries.  The  goal  is 
to  make  computers  more  situation-aware  by  designing  irmovative  algorithms  that  infer 
meaning  from  the  way  people  arrange  information  spatially.  EWall  uses  Information 
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Objects  (indeed,  our  concept  of  lOBs  originated  from  the  EWall  project)  created  and  sent 
automatically  to  EWall  agents  that  convert  user-generated  spatial  information 
arrangements  into  networked  information.  EWall  uses  this  information  to  adjust  its 
selection  criteria  and  presents  users  with  increasingly  relevant  information  as  it  learns. 
EWall  will  consist  of  five  independent  frameworks  (modules)  addressing  novel  issues  in 
Information  Visualization,  Communication  and  Management. 

Of  particular  interest  to  our  project  are  the  Communication  and  Visualization  modules. 
The  Communication  Module  will  determine  differences  and  similarities  among  the  users’ 
individual  Work  Areas.  Based  on  this  analysis  this  Module  will  be  able  to  automatically 
identify  groups  of  users  that  pursue  similar  lines  of  inquiry,  detect  emergent  directions 
and  sub-tasks,  and  prioritize  the  information  exchange  between  multiple  users.  The 
Visualization  Module  will  offer  various  ways  of  viewing  and  rearranging  the  content  on 
Work  Areas  and  from  other  information  sources.  This  Module  will  contain  a  wide  variety 
of  algorithms  to  automatically  rearrange  Information  Objects  as  well  as  to  add  and 
modify  previously  established  relationships  among  Information  Objects.  The  algorithms 
will  respond  to  the  users’  different  styles  of  working  and  allow  users  to  explore  various 
new  relational  aspects  from  the  information  sources.  We  expect  the  availability  of  these 
algorithms  will  significantly  improve  the  ability  of  group  members  to  make  both 
individual  and  overall  group  quality  decisions.  It  should  also  minimize  the  time  devoted 
to  conflict  resolution  and  consensus  building 

Figure  5  shows  a  schematic  representation  of  the  overall  EWall  project,  including  the  five 
basic  modules.  Figure  6  shows  a  sample  display  from  the  Visualization  module. 
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Figure  5:  EWall  Overall  Structure 


T  ransfarmatfin  Vra^^^ 


Canlext  View 


Figure  6.  EWall  Visualization  Module  Sample  Display 


4.0  Summary: 

Three  deficiencies  were  addressed  in  this  paper:  (1)  the  difficulty  group  decision  makers 
have  in  sharing  their  uniquely  held  information  (2)  the  difficulty  they  have  in  integrating 
such  information  into  their  on-going  decision  processes  if  it  actually  does  become 
available  and  (3)  the  difficulty  they  have  in  conflict  resolution  and  consensus  building, 
particularly  in  asynchronous  collaboration. 

Our  primary  hypothesis  is  that  a  major  culprit  in  all  of  these  deficiencies  is  the  cognitive  burden 
the  tasks  place  on  both  the  sender  and  recipient  of  the  information  being  exchanged.  In  the 
decision  making  task  presented  in  this  paper,  the  information  item  is  typically  a  text  document, 
i.e.  a  company  report,  CNN  web  page,  email,  military  message  traffic,  etc.  These  documents  can 
vary  on  a  number  of  parameters  and  it  is  difficult  to  quickly  assess  an  “overall  picture”  of  the 
content,  validity  and  timeliness  of  the  item.  We  propose  this  issue  be  addressed  by  using  a  user- 
friendly  on-line  knowledge  elicitation  technology  that  assesses  the  participant’s  perception  of  the 
critical  parameters  of  each  item  he  wishes  to  retain.  This  information  is  then  automatically 
converted  into  a  small  color-coded  icon  called  an  information  object  (lOB).  The  lOB  then 
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becomes  a  simplified,  easily  categorized  graphical  representation  of  the  complex  text  item.  The 
on-line  sharing  of  these  lOBs  among  the  participants  obviates  the  problem  of  sharing  unique 
information,  and  the  pre-assessed,  compact  nature  of  the  lOB  information  significantly  increases 
the  likelihood  that  the  contents  will  be  factored  into  a  recipient’s  on-going  decision  process.  The 
attached  subjective  cognitive  scoring  also  serve  as  a  precise  focus  point  for  reconciling  the 
different  assessments  and  perceptions  of  the  impact  of  the  item,  substantially  reducing  the 
discussion  time  that  needs  to  be  devoted  to  conflict  resolution  and  consensus  building. 
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•  Sharing  of  information  is  difficult: 

-  Sender  doesn’t  know: 

•  Who  knows  what 

•  Who  needs  what 

•  Processing  shared  information  is  difficult: 

-  Recipient  has  significant  cognitive  burden  just 
processing  the  information  he  finds 

•  Information  received  from  other  participants  is 
therefore  often  discounted/discarded 


Sample  Decision  Task 


V 

Syst&ms  Center 
San  Dfego 


•  Four  staff/intelligence  personnel  asked  to  assess  Islandia  as  site 
for  a  refugee  camp.  Make  one  of  these  recommendations: 


feutral 


Strong 

Negative 


Strong 

Positive 


Negative  Positive 


•  Base  recommendation  upon  these  criteria: 

-  Islandia’s  Infrastructure  (INF) 

-  Available  Labor  Pool  (LAB) 

-  Local  Security  Force  (SEC) 

-  Impact  on  our  Resources  (RES) 


•  Group  is  Distributed  (D.C.  to  Japan) 


Recommendation  needed  by  1600  tomorrow 


Decision  Sequence  (member  M2) 


jsFmm 

Syst&ms  Center 
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Group’s  Final 
Recommendation 


Confiict  Resolution, 
Consensus  Buiiding 


M1  M3  M4 


0 


M2 


Islandia  Site 


0 


Overall  Assessment 


Score  INF  Criterion 


Score  LAB  Criterion 


A 

A 

A 

Collect/Evaluate  INE  data 

Collect  /Evaluate  LAB  data 

Collect/Evaluate  SEC  Data 

\ _ 

J 

J 

Score  RES  Criterion 

I 

Collect/Evaluate  RES  data 


0^ 


!^f 


An  Analyst’s  Review  &  Selection  of  srmm 


Information  Items 
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No 


No 


Discard 


Search 

databases 


Item  relate  to  a 
deeision 
criteria? 


Usable  Item? 


Make 
item 
evaluations 

Store 
Item  in 
Cognitive 
Memory 


What  a^ut  the  information  t^elevanej^ 

Does  it  impact  one  of  my  decision  \ 
criterion? 


What  about  the  information|Quality^ 

KEEP 


What  is  the 


Effect  [pf  the  informarion? 


EVALUATE 


Cognitive  Abstractions 
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•  The  analyst  has  performed  a  set  of  cognitive 
information  abstractions  on  each  retained  item. 


•  These  abstractions  are  not  formalized  or  made 
explicit  but  rather  remain  in  cognitive  memory. 

-  They  are  not  available  for  exchange  between  group 
members 

-  What  is  exchanged  are  the  information  items,  not 
the  information  abstractions. 


Hypothesis: 
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•  One  approach  to  the  reduction  of  this  cognitive  burden 
would  be  to: 

translate  the  cognitive  abstractions  into  a  structured 
graphical  representation. 

•  Expected  Result: 

-  Formalize/standardize  the  abstraction  process 

-  The  abstracted  information  could  be  encapsulated  into  readily 
understandable  and  transferable  Iconic  symbology,  resulting  In: 

-  More  Information  being  exchanged  and  used 

-  Improved  individual  and  group  decision  making 

-  Faster  conflict  resolution/consensus  building 


Group  will  have  significantly  improved  Shared  Mental  Model  of  the  situation. 


-S79IMW7 

“Moving  abstractions  from 
cognitive  memory  to  computer  memory” 
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Issues  to  be  addressed: 


(1)  What  parameters  need  to  be  abstracted? 

(2)  How  do  we  accomplish  this  formalized 
abstraction? 

(3)  How  should  we  encapsulate  (display)  the 
results  to  optimize  information  exchange  and 
integration? 


Information  Parameters 
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•  Where  does  it  belong? 

-  Assign  to  a  Decision  Criterion. 

•  What  about  the  Quality  of  the  Information? 

-  Credibility,  Timeliness,  Confidence/Uncertainty 

•  Effect  of  content  on  Criterion? 

-  Positive/Negative,  Major/Minor 

•  How  Important  is  this  item? 

•  Administrative 


(2)  The  Abstraction  Process 
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•  #1  REQUIREMENT:  MINIMUM 
COGNITIVE  BURDEN. 

•  Right-hand  mouse  click  presents 
abstraction  template  on  screen. 

•  Required  Parameters  are  listed 
sequentially 

•  Option  selections  are  color-coded  and 
selected  by  a  radio-button  click. 


The  abstraction  process 
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This  impacts  Infrastructure,  I  wish  to  retain  it. 
(Right-hand  mouse  click) 
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Abstraction  Template  Design 
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Based  upon  cognitive  assessment  sequence 


Quality 


TITLE: 


Criterion: 
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Using  the 

Abstraction  Template 
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“The  Asian  Times  reported  yesterday  that  unidentified  sources 
claim  that  the  completion  of  the  runway  extension  at  Islandia’s 
main  airport  will  be  delayed  at  least  one  week.” 


TITLE:  runway  delay 

Criterion:  ^IKER.  OlABOR  0  SECURITY  OreSOURCES 


Credibilitv 


Timeliness 


Confidence 


o 
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(3)  Encapsulating  the  Abstractions 

Encapsulation  Guidelines: 

•  For  browsing/search,  the  title/keyword 
should  be  up  front  and  readily  visible. 

•  The  most  important  parameters  are: 

-  The  Effect  on  the  criterion 

•  Direction:  Positive  or  Negative 

•  Degree:  Minor  or  major 

-  The  Importance  of  the  item 

•  Average,  High  or  Very  High 
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Secondary  parameters  are: 

-  Quality  of  Information 

-  Administrative  tags 


Creating  an 

Information  Object  (lOB) 
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Abstraction  &  Encapsulation 
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TITLE: 

Criterion:  %mER.  OlABOR  O  SECURITY  OreSOURCES 


I  AD  SEC 
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“Tag. 


.and  Release 
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•  Sort  by  Importance: 

4  out  of  5  items  rated  as  having  High 
importance  have  a  negative  impact  on 
this  decision  criterion  . 


What  is  the 
assessment? 
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Another  lOB  Advantage: 
Remember.... this  is  Group  Decision  Making 
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Final 

Recommendation : 


Need  for  Conflict  Resolution/Consensus  Building 


Focused  Discussion  of  Differences 
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lOBs  permit  FOCUSED  discussion,  e.g. 

‘‘Let’s  discuss  the  credibility  and  timeliness  of  Item  #26” 
vs.  an  open-ended  “Why  did  M3  score  criterion  INF  so  high?” 


overriuis  are 
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:  delays 
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Focused  discussions  are  essential  in  the  austere  (email,  chat 
box,  etc.)  environment  of  asynchronous  collaboration 


Collaborative  Decision  Sequence 
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•  Our  Current  Research  is  directed  at: 

-  Methodologies  for  transferring  implicit 
cognitive  assessments  into  explicit  graphical 
representation. 

-  Experimental  evaluation  of  their  effect  upon 
the  sharing  and  integration  of  decision¬ 
relevant  information  in  a  collaborative, 
asynchronous  group  environment. 

-  Do  we  get  better  quality  decision  making? 
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